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Comment on "Limits on the Time Variation of 
the Electromagnetic Fine-Structure Constant..." 

In their Letter 1} (also @), Srianand et al. analysed 
optical spectra of heavy-elements in 23 absorbers along 
background quasar sight- lines, reporting limits on varia- 
tions in the fine-structure constant, a: Aa/a — (— 0.06± 
0.06) x 10~ 5 . This would contradict previous evidence 
[e.g. H, 0| for a smaller a in the absorption clouds com- 
pared to the laboratory: Aa/a = (-0.57 ±0.11) x 10~ 5 
5|. Here we demonstrate basic flaws in the analysis of 
l| using the same data and absorption profile fits. 

For each absorber, Aa/a is measured using a % 2 mini- 
mization of a multiple-component Voigt profile fit to the 
absorption profiles of several transitions. The column 
densities, Doppler widths and redshifts defining the com- 
ponents are varied iteratively until the decrease in x 2 be- 
tween iterations falls below a specified tolerance, Ax 2 ol . 
In our approach, we simply add Aa/a as an additional 
free parameter whereas [l| keep it as an external one: for 
each fixed input value of Aa/a the other, free parameters 
are varied to minimize % 2 . The functional form of % 2 im- 
plies that, in the vicinity of the best-fitting Aa/a, the 'x 2 
curve' - the value of \ 2 as a function of Aa/a - should 
be near parabolic and smooth. That is, A\ 2 ol should be 
<C 1 to ensure that fluctuations on the x 2 curve are also 
■C 1. This is crucial for deriving the l-a uncertainty in 
Aa/a from the width of the x 2 curve at x^in + 1- 

However, none of Srianand et al.'s x 2 curves - figure 
2 in [l|, 14 in Q - are smooth at the <C 1 level; many 
fluctuations exceed unity. Two examples are reproduced 
in Fig. [TJ The fluctuations can only be due to failings in 
the x 2 minimization: even when ^2] fit simulated spec- 
tra (their figure 2) jagged x 2 curves result, leading to a 
strongly non-Gaussian distribution of Aa/a values and a 
large range of l-a uncertainties (their figure 6). Clearly, 
these basic flaws in the parameter estimation will yield 
underestimated uncertainties and spurious Aa/a values. 

To demonstrate these failings, we apply the same pro- 
file fits to the same data but with a robust x 2 mini- 
mization. The spectra were kindly provided by B. Aracil 
who confirmed that the wavelength and flux arrays are 
identical to those in [l[. For each absorber, the best- 
fitting profile parameters of 0] were treated as first 
uesses in our x 2 minimization procedure (detailed in 
). The relationships between the Doppler widths of 
corresponding velocity components in different transi- 
tions were also the same, as were the relevant atomic 
data. The relative tolerance for halting the x 2 minimiza- 
tion was Ax 2 ol /x 2 — 2x 10~ 7 . All absorbers yield smooth 
X 2 curves in new our analysis; Fig. [T] shows two examples. 
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FIG. 1: Example \ 2 curves from our minimization (circles) 
and that of [l[ (triangles). Fluctuations in the latter indicate 
failings in the minimization. Points and error-bars indicate 
best-fitting values and l-a uncertainties; for our curves Aa/a 
was a free parameter. Note the different vertical scales: left- 
hand scales for our curves, right-hand scales for 



By-products of this analysis are revised values of Aa/a 
and l-a errors. We find 14 of the 23 Aa/a values devi- 
ate by > 0.3 x 10~ 5 from those of [l[. Moreover, the 
errors are almost always larger, typically by a factor of 
^3. The formal weighted mean over the 23 absorbers be- 
comes Aa/a = (-0.44 ± 0.16) x 10~ 5 but the scatter in 
the values is well beyond that expected from the errors. 
This probably arises from many sources, including overly 
simplistic profile fits (see 0). Allowing for additional, 
unknown random errors by increasing the error-bars to 
match the scatter (i.e. xt = 1 about the weighted mean), 
a more conservative result from the data and fits of [l[ is 
Aa/a — (—0.64 ± 0.36) x 10~ 5 - a 6-fold larger uncer- 
tainty than quoted by [l[. We conclude that the latter 
offers no stringent test of previous evidence for varying a; 
this must await a future, extensive statistical approach. 
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Discussion of Srianand et ai.'s Reply to our Com- 
ment 

Our Comment (arXiv:0708.3677) sought to demon- 
strate that the results of Srianand et al. (also 0) 
were not robust and were based on a measurement tech- 
nique which failed in a fundamental way. The numbers 
emerging from the failed algorithm arc meaningless, as 
discussed again below; they cannot even be considered 
approximately correct. Indeed, when we apply the same 
measurement technique (without the failure) to the same 
data, using the same profile fits as [3, 0], we find very 
different values of Aa/a and errors which are typically a 
factor of ~3 larger. We present much more detail of that 
analysis in 

In their Reply (arXiv:0711.1742), Srianand et al. state 
or argue several points, all of which we dismiss below ei- 
ther because they are demonstrably incorrect or because 
they rely on a flawed application of simple statistical ar- 
guments. In order of importance: 

Point (i): Despite demonstrating in our Comment 
that the measurement procedure used by [H, 0] failed, 
their Reply argued that indeed their "procedure is ro- 
bust as shown in" Q . Much of the discussion and Fig. Q] 
in our Comment - and, indeed, Srianand et a/.'s own 
figures (2 in [l[, 14 in - demonstrate the precise op- 
posite, i.e. that large fluctuations on the \ 2 curves are 
clearly present. This means that at any given point on a 
X 2 curve (i.e. for a given input value of Aa/a), the true 
minimum of % 2 cannot have been reached. Therefore, the 
true x 2 curve must lie entirely beneath that derived by 
[uka] for most absorbers. That is to say, the x 2 curves of 
[l|, |2J simply cannot be used to infer any values of Aa/a 
or their 1-a errors whatsoever. Nor can values of Aa/a 
(or uncertainties) one chooses to infer from them even 
be considered 'approximately correct' in any meaningful 
way. These basic and fundamental aspects of x 2 fitting 
cannot be overemphasized: x 2 curves with large fluc- 
tuations provide no meaningful measurement of 
any kind. 

In their Reply, Srianand et al. also "point out that 
fluctuations in x 2 curves get indeed smoothed after a 
large number of iterations but the results from the first 
and last iterations are found to be very similar" . It must 
again be strongly emphasized that fluctuations on a x 2 
curve indicate nothing but the simple fact that x 2 has not 
been reduced to its true minimum value at some, if not 
all points on the curve. One can not "smooth" these fluc- 
tuations in any way not by averaging many "iterations" 
together (as may be implied by the above statement), nor 
by fitting a parabolic or polynomial line through the x 2 
curve as one would fit a model to noisy data. It is not 
"noise" at all, but just an indication that the algorithm 
for reducing x 2 has failed. One simply has to identify 
the coding error, bug or mis-use of the algorithm which 
is preventing x 2 from reaching its minimum. 

Srianand et al. [l], 0] also argue that their measure- 



ment procedure is robust based on measurements using 
simulated absorption systems and these are referred to 
again in their Reply. However, as we pointed out in our 
Comment (and in more detail in Q), those simulations 
actually demonstrate the precise opposite: strong fluctu- 
ations even appear in the x 2 curves for these simulations 
(figure 2 in [2 1). This leads to spurious Aa/a values: 
figure 6 in 0] shows the results from 30 realizations of a 
simulated single-component Mg/Fe n absorber. At least 
15 Aa/a values deviate by > ler from the input value; 8 
of these deviate by > 2a and 4 by > 3a. There is even 
a w 5-fi value. The distribution of Aa/a values should 
be Gaussian in this case but these outliers demonstrate 
that it obviously is not. The x 2 fluctuations also cause 
the uncertainty estimates from the different realizations 
to range over a factor of « 4 even though all had the 
same simulated spectral signal-to-noise and input profile 
fitting parameters. None of these problems arise in our 
own simulations of either single- or multiple-component 
systems (see [1] for detailed discussion). 

Point (ii): We used smaller error spectra in our Com- 
ment compared to the original analysis of [H, 0] , as de- 
scribed in detail in [6J. Although Srianand et al. point 
out this fact in their Reply, they do not discuss its import. 
We do so in 'fi}. To summarize: The main argument in 
our Comment is that the error bars of [J, 0] are under- 
estimated. The fact that we used smaller error arrays 
and still found much larger errors on Aa /a than [l], 0] 
only emphasizes this argument more. Using somewhat 
larger error arrays would simply lead to somewhat larger 
uncertainties on Aa/a and change the actual measured 
values of Aa/a negligibly. Thus, "Point 2" in the Reply 
actually reinforces our conclusion that the data and fits 
do not offer a stringent test of previous evidence for a 
varying a. 

Point (iii): The Reply argues that many of our revised 
values of Aa/a "match" the original values of Q "at 
< la level" and, therefore, that the original results are 
robust. Three simple points can be made here; the first 
two (1 & 2) are practical while the third (3) is a more 
important general one: 

1. A matching criterion of "< lc" is used for each 
value of Aa/a, but it isn't clear if "cr" is the orig- 
inal uncertaintiy from [l], 0] or our revised value 
which is typically larger by a factor of ^3. More 
importantly we cannot replicate the number of sys- 
tems which supposedly "match" , i.e. 16. Using the 
uncertainty values from 0, 0] as "cr" we find 11 
'matching' values of Aa/a. Using our new uncer- 
tainties gives 13 'matches'. Finally, adding the two 
uncertainty values in quadrature and treating that 
as "cr" gives 14 systems. 

2. Even if the old and new Aa/a values were statis- 
tically independent, how many values would one 
expect to "match" given the general distribution of 
values and uncertainties in both measurement sets? 
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Whatever that number, even more values would 
"match" when we consider that the new and old 
values are in fact correlated. Therefore, at least 
some of the 16 'matches' the Reply cites cannot be 
argued to bolster a case for the robustness of ei- 
ther the old or new values. Only if one found many 
more matches than one expects, given the two sets 
of values and their errors, would there be some case 
for that. 

3. The more important general point is that the usual 
"1 ct" threshold is meaningless here because the 
new and old Aa/a values are not independent in 
any respect. When such non-independent data are 
being compared, it should actually be very few val- 
ues (not 32% as for truly independent values) which 
would deviate by more than one formal standard 
deviation. Only if the deviation was very small 
with respect to the (old or new) 1-a uncertainties 
could one claim that the faults in the x 2 minimiza- 
tion algorithm had negligible importance and that 
the original results were robust. The large devia- 
tions we observe - 14 of the new 23 Aa/a values 
deviate from the old by > 0.3 x 10 -5 - clearly imply 
that the old results were not robust at all. 

Point (iv): The Reply discusses how two Aa/a values 



in our revised set deviate by more than 4 a (presumably 
with respect to Aa/a = 0) and Srianand et al. choose to 
remove them from the sample. How can one decide which 
systems are best to remove based only on the parame- 
ter of interest, in this case the value (or significance) of 
Aa/a? This is obviously a very biased selection method. 
In our Comment, this is why we chose to increase the er- 
ror bars on all Aa / a values to match the observed scat- 
ter. Although itself not ideal, this is at least an unbiased 
procedure. Our main point remains whatever procedure 
one chooses: the constraints obtained from the data and 
fits of [l|, [2| can not be regarded as robust, nor are they 
a stringent test of previous evidence for varying a. 

Point (v): Srianand et aZ.'s Reply states that their 
"procedure takes into account the differences in spec- 
tral resolution in different settings ... while this is 
not the case with vpfit". This is simply incorrect, 
as is easily verified by reading the vpfit manual at 
http://www.ast.cam.ac.uk/~rfc/vpfit.htmll [for the cur- 
rent versions (8 & 9), this is mentioned in the first few 
lines of the introduction] . vpfit has always has this fea- 
ture. Differences in resolution between different portions 
of input spectra were taken into account in all previous 
analyses of Keck/HIRES spectra in 0, 0, Q using vpfit. 



